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ABSTRACT 
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The Government Performance and Results Act of 1993 (GPRA) is 
legislation passed in response to growing concern over the efficiency and effectiveness 
with which the federal government performs its functions. Under GPRA, all federal 
agencies will be required to submit annual performance plans and reports starting in 
September of 1997 for the FY99 budget request. As a major agency, DoD will be 
required to submit these performance plans and reports. The act requires pilot projects 
as a test of f)erformance planning and reporting. 

This thesis is an analysis of the performance plan/report pilot project 
currently in progress at the Defense Logistics Agency. Included is an examination of the 
two performance plans already submitted by DLA. This thesis analyzes the performance 
measures used, the performance measurement system, and the performance plan format. 
Central to the thesis is an evaluation of the process involved in implementing GPRA by 
the employees and managers of DLA, 

Other data provided is a definition of performance budgeting, 
definitions of various performance measures, a history of budget reforms which used 
performance measures as a means of improving government programs and finally, an 
analysis of the other DoD pilot project performance plans. 
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I. INTRODUCTION 


A. BACKGROUND 

Confronted by the twenty-sixth straight year of deficit 
spending and a four trillion dollar national debt, 
governinent officials seem increasingly aware of the 
declining public tolerance for such conditions. This has 
led to several attempts in recent years at reducing annual 
budget deficits. Most notable among these were the Gramm- 
Rudman-Hollings Act and amendment in 1985 and 1987 
respectively and the Budget Enforcement Act of 1990 and 
1993. While the Budget Enforcement Acts resulted in 
promises of nearly $ 1 trillion in deficit reduction, all 
four measures have failed to produce the balanced budget 
originally sought (Berner and Daggett, 1993, pp.39-40). The 
failure of the federal government to reconcile this problem 
has led to a question of the effectiveness and efficiency 
with which the federal government operates. 

This interest in how the federal government performs 
its various activities has in turn resurrected the concept 
of performance measurement and its possible effects on the 
budgetary process. At least four different efforts have 
stimulated a move toward performance measurement; these 
include the application of financial reforms for federal 
management included in the Chief Financial Officers Act, the 
National Performance Review led by Vice President Gore, the 
1992 Defense Authorization Act and the Government 
Performance and Results Act of 1993 (GPRA), which called for 
the creation of federal agency performance plans. 

The Chief Financial Officers Act of 1990 requires the 
designation of a chief financial officer (CFO) in each of 
the major executive agencies of the federal government. 

This act charges the CFO, as one of his primary duties, to 



"develop and maintain an integrated agency accounting and 
financial management system ... which - provides for - the 
systematic measurement of performance . (P.L. 101-576, 1990) 
The act does not specify how or what to measure, but does 
make performance measurement a primary goal. 

Secondly, the National Performance Review concluded the 
government should use budgeting as a means of improving 
program effectiveness and results . It urged the President 
to negotiate performance agreements with agency heads, and 
specify what should be accomplished with the given resources 
over the next several years. (National Performance Review, 
1993) Those critical of the National Performance Review say 
the report is clear about what it is against, but becomes 
vague when it suggests how to fix these areas. Nonetheless, 
the report established the importance of performance 
measurement in the budget process as a major milestone for 
government improvement. 

Thirdly and most recently. Congress has passed two 
pieces of legislation which deal with performance 
measurement. The Fy93 "National Defense Authorization Act" 
under subtitle D requires the Secretary of Defense to 
" develop performance measures and corresponding performance 
goals for each business area..." of the Defense Business 
Operations Fund (DBOF)(P.L. 102-484, 1992). Performance 
measurement and budgeting seem appropriate in this endeavor 
since the Department of Defense is attempting to more 
closely model private industry in its operations and control 
of the DBOF. 

On August 3rd, 1993 Congress passed P.L. 103-62, the 
Government Performance and Results Act (GPRA). The purpose 
of the act is to shift the focus of government management 
from inputs to outputs and outcomes, from process to 
results, from compliance to performance, and from management 
control to managerial initiative. The act requires all 



federal agencies to define long term goals, set annual 
performance targets derived from these goals, and annually 
compare actual performance to the targets. The act also 
establishes three sets of pilot projects within the federal 
government: performance plans and reports, enhanced 
managerial flexibility and accountability, and performance 
budgeting. (P.L. 103-62, 1993) In short the act requires a 
significant change in the way we think about management and 
budgeting in the federal government. 

These four different initiatives have shown the 
significant increase in concern over performance issues 
within the federal government. This thesis will concentrate 
primarily on the implementation of the GPRA within the 
Department of Defense (DoD). The GPRA was chosen over the 
others, because it requires a systematic approach to the 
performance issue. The next section will describe the focus 
of the thesis. 

B. THESIS SCOPE AND APPROACH 

The objectives of this thesis are to: 

• examine the background of performance measurement 

• compare several DoD pilot projects under the GPRA 

• examine the process used for GPRA performance 
measurement at the Defense Logistics Agency (DLA) 

• to assess how GPRA performance measurement might be 
implemented throughout the DoD 

DLA was chosen for the detailed case study analysis 
because it was one of the first pilots selected and because 
the management of DLA had volunteered to do so. This 
establishes a strong desire by the agency to make the GPRA 
work and thus a qualified model to investigate. 




Chapter II provides the legislative history of the 
Government Performance and Results Act in 1993. 

Additionally, several different measures of performance will 
be reviewed along with a definition of performance 
budgeting. The budget definition is included since the 
premise of GPRA is that some day performance budgeting will 
be used as the primary means for resource allocation within 
the federal government. 

Chapter III will present a history of budget reforms 
over the past century. These reforms, like GPRA, also 
attempted to use performance measurement as a means of 
improving federal government performance. 

Chapter IV will be an analysis of the DoD 
implementation of the GPRA. Specific emphasis will be given 
to the seven pilot projects currently in progress under the 
auspices of the GPRA. A comparison of the performance 
measures used will be provided. A look at how DoD is 
implementing GPRA on the corporate level is also provided. 

Chapter V will be a detailed analysis of the Defense 
Logistics Agency's attempt at implementing the GPRA. This 
chapter is based on interviews with several individuals at 
the corporate headquarters and at field activities. The 
discussion focuses on answering the below questions within 
the context of the DLA pilot project. In the case study, 
the following questions will be researched: 

• How were performance measures captured? 

• How did the process of measurement work in the eyes 
of the individuals involved? 

• What are the strengths and weaknesses of the system? 

• What were the expectations for how the process would 
work? How did the system work? 



• How can performance measures be used to complement a 
budget? 

• How can this information be used to help the DoD and 
the Don develop future performance measures and 
budgets? 

Chapter VI contains the conclusions reached in this 
study. Suggestions and recommendations for implementation 
of the GPRA within the DoD and DoN will be provided. 




II. GPRA AND PERFORMANCE MEASUREMENT 


A. LEGISLATIVE HISTORY OF GPRA 

This section of the thesis will detail the more recent 
legislative history leading up to the GPRA. On October 3, 
1990, Senator William Roth introduced S.3145, the Federal 
Program Performance Standards and Goals Act of 1990. The 
101st Congress decided not to take action on the bill during 
that particular session. This however, proved to be the 
start of a resurgence of interest in federal government 
performance measures. S.20 was introduced at the start of 
the 102nd Congress in the exact same form as it had been in 
the previous Congress. Two hearings were held by the Senate 
Committee on Governmental Affairs where the bill was 
sponsored by Senator John Glenn, chairman of the committee. 

At Senator Glenn's request, several studies were 
performed to gain some insight into the uses of performance 
measurement and budgeting. Two important studies in this 
area were completed by the General Accounting Office. The 
first, entitled "Program Performance Measures, Federal 
Agency Collection and Use of Performance Data" was an study 
to gauge the current use of performance measures throughout 
the federal government. The second, entitled "Performance 
Budgeting, State Experiences and Implications for the 
Federal Government" was a study of state experiences with 
performance measurement and budgeting. Both studies 
discovered that in general performance measures were not 
used as often as respondents had actually claimed and that 
performance budgets were not used in a pure format. Still 
the 102nd Congress did not act to pass the bill into law. 

During the 103rd Congress, S.20 again was introduced. 
This time, however, a few changes had been made and the name 
was shifted to the Government Performance and Results Act. 

A similar bill was also introduced in the House by 





Representative John Conyers, Chairman of the Government 
Operations Committee. His bill, H.R. 826, was nearly 
identical to the Senate version. Committee hearings were 
held on both bills and they eventually made it to the full 
House and Senate for vote. H.R. 826 passed the full House 
on 25 May 93 by unanimous consent. Likewise, S.20 passed 
the full Senate on 23 June 93 also by unanimous consent. 

The House approved the Senate version on 15 July 93 clearing 
the way for the bill to become law. The Government 
Performance and Results Act of 1993 became law on 3 August 
93 with the signing by President Clinton. A press 
conference was held during the signing at which the 
President held high hopes for the new law by exclaiming: 

...to the extent that our government works with 
greater efficiency and effectiveness and less 
unnecessary cost, it will strengthen the American 
economy as well as the bonds of our citizenship. 

(Office of the Press Secretary, 1993) 

Table 1 spells out briefly the requirements for performance 
measurement and budgeting contained in the GPRA of 93. 

As can be seen by Table 1, performance plans (i.e., 
performance measures) will be required by all federal 
agencies in 1997. Also occurring in 1997 will be the 
selection of a few agencies to serve as pilot projects in 
performance budgeting. The next section of the thesis will 
describe performance budgeting since this is the ultimate 
goal of GPRA. Along with the budget definition, several 
different forms of performance measures will be defined and 
illustrated. Additionally, some examples of budgets 
containing performance data will be used to illustrate what 
some government agencies have used in the past. 
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Table 


GFRA Requirements 


1993 


1994 


1997 


1998 


2000 

2001 


Source: 


_ OBJECTIVE _ 

Selection of at least ten agencies as pilot sites for 

testing performance plans and reports. _ 

At least five of these agencies selected as pilot sites 
__ for testing managerial accountability/flexibility. _ 

OMB reports to Congress on pilot results, GAO reports to 

__ Congress on readiness for full implementation. _ 

All Federal agencies submit 5-year strategic plans. _ 

All Federal agencies submit annual performance plan for 

FY99. _ 

Selection of at least 5 pilots for performance budgeting. 
OMB submits gov't wide performance plan for Fy99 to 
_ Congress ■ _ 

Agencies submit annual performance reports for FY99. _ 

OMB reports to Congress on pilot test of performance 
budgeting. 

P.L. 103-62, 1993 


B. PERFORMANCE BUDGETING AND MEASURES DEFINED 

Performance budgeting requires an agency to develop 
objectives or goals which it desires to accomplish. The 
cost of the programs required to perform those objectives 
and quantitative data measured to ensure the objective is in 
fact accomplished in an effective and efficient manner are 
also important. The key objective of performance budgeting 
itself is to produce a financial and managerial atmosphere 
that will assist managers with appropriate information and 
accountability to reduce costs while achieving objective 
levels of performance (Shycoff, 1992). 

The following is a generally accepted prescription for 
the implementation of program and performance budgeting: 






















• Identify programs and program costs. 

• Break down the programs into smaller units called 
sub-programs, activities and cost centers. 

• Develop units of measurement for the work to be done 
for each activity. 

• Measure the work to be done for each activity and the 
time and cost required to accomplish this work. This 
requires a work measurement system and an accounting 
system that accumulates these values. 

• Develop unit times and unit costs for each activity, 
(historical data or standards should be used). 

• Summarize all the data in a work program for each 
activity: work to be done, units of measurement, unit 
time and cost, total time and cost, etc. 

• Use the work program for each activity or cost center 
to develop the agency budget. 

• Appropriate funds by program. 

• Report and compare actual performance and cost with 
projected values. 

• Use quarterly performance reports in budget 
implementation as a basis for allocating funds to 
agencies. 

• Determine the extent to which the achievement of the 
targets for each activity led to the attainment of 
the short and long term program goals. These are 
measures of effectiveness, quality and results. 

(Source; Axelrod, Budgeting for Modern Government, pp. 267) 


As can be seen by this list, performance budgeting 
focuses on efficiency accountability. In other words, this 
type of budget aims to assist managers in wisely spending 
money such that maximum output is achieved with as little 
input as possible. Notice also the focus shifts from 
objects of expenditure to program activities as the basis 
for budgeting. Therefore, instead of budgeting for 
salaries, utilities, and travel expenses; the manager would 
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base the budget upon educational activities, regulation, 
research, executive direction, etc. (Wanat, 1978, pp. 95-98) 

Once the programs have been bro)cen down into 
activities, measurements for performance must be generated 
for program evaluation. The Comptroller of the DoD has 
defined five generic performance measures to be implemented 
in context with performance budgeting. (Shycoff, 1992) These 
measures are input, workload, efficiency ("doing the thing 
right"), effectiveness ("doing the right thing"), and 
impact/outcome measures. Figure 1 is an example of some of 
the performance measures used by the Defense Logistics 
Agency (DLA) in gauging the performance of their 
distribution function. This Figure will be referred to as 
each type of measurement is explained. Input measures 
describe the resources, time, and personnel used for a 
program. Such items as appropriated dollars, end strength, 
and staff training hours, fit into this category. 

If no other type of measure is provided, then as a bare 
minimum some measures of workload should be presented. Such 
items as number of checks issued, number of arrests made, or 
licenses issued would suffice as measures of workload. In 
the DLA example, no measure of workload is presented. This 
is not bad, since workload measures are really the lowest 
form of performance measurement desired. While workload 
measures do describe the activities of a program, they do 
not really define how well the program is accomplishing its 
mission. 

Measures of efficiency take the workload data and merge 
it with cost data in order to develop unit cost measures. 
Then efficiency can be gauged on such items as cost per 
arrest made, cost of issuing a check, cost of flying an 
aircraft per hour, etc. In the DLA case, the "Depot Line 
Rate" is the charge to an inventory control point per unit 
of distribution work performed (e.g., lines received. 
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issued, etc.)- Efficiency is a much better indicator of 
performance than simple workload data since it gives outputs 
a direct cost relationship. These costs per unit can then 
be compared over time or against other similar activities to 
gauge competitiveness or improvement. This is extremely 
important since it allows administrators a simple way to 
keep track of complex programs at a smaller level, Congress 
can track efficiency measures to keep costs down, and the 
public can be assured its taxes are being spent effeciently. 

Effectiveness measures are used to mark output 
conformance to specified characteristics. Such items as 
quality, timeliness, and customer satisfaction fall into 
this particular category of measures. In Figure 1, 

"Material Release Order Processing Time" and "Customer 
Satisfaction" are examples of effectiveness. These measures 
require the managers to determine goals for the particular 
program activity. Moreover, these types of measures require 
managers to identify who their customers are and what type 
of characteristics customers would desire within the 
products. Effectiveness measures are better than efficiency 
measures in that the primary focus of effectiveness is on 
the customers, whereas the primary focus of efficiency is 
the organization. 

The last type of measure to be looked at within a 
performance budget is the outcome,impact or result. This 
measure has proved to be the most difficult to develop. 
Outcome measures attempt to capture performance on the basis 
of achieving what the program desired to do as a whole. 
Simply put, did the program achieve the mission it set out 
to do from the start? 

In the case of the DLA measures, no true measure of 
outcome is provided. DLA's mission is to provide the U.S. 
armed forces with the appropriate level of materials in 
order to carry out a military action. Therefore, the only 
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true measure of outcome would be to wait for a military 
action and see if the U.S. services were provided with the 
material required. Short of war managers must develop the 
closest thing to outcome measures they can. This might mean 
establishing an intermediate outcome. For example, a cancer 
treatment facility may have as its primary mission to reduce 
the reoccurrence of cancer in its patients. In order to 
gauge success, it might attempt to track previous patients 
who develop cancer again at a later date, then compare this 
number to the total number of patients it has treated. 

Finally, a "pure" performance budget should consist of 
activity classifications, workload data, other measures of 
performance, unit costing data, and program goals. Other 
data typically found in budgets which are modeled after 
performance budgets consist of narratives discussing the 
activity or program, several years' worth of data, mission 
statements, and outcomes desired. It should be noted most 
budgets which are called performance budgets are not of the 
pure format. The examples in Figures 2 and 3 are budgets 
which contain performance data but still retain traditional 
data as well. Thus they are a hybridization of performance 
and traditional budgets. It can be observed that in both 
cases, the traditional object of expense is present as well 
as the performance indicators. Measurements used in these 
budgets consist of unit cost data (or efficiency), such as 
the "annual per capita cost" for the Patuxent Institute, and 
workload data, such as the "man-days required to lay patch 
with premix" in the highway budget. Notice also the rather 
extensive narrative associated with the Patuxent 
Institution's budget. Notable by their absence are measures 
for effectiveness and outcomes. 

This section has given a definition of performance 
budgeting and different types of performance measures 
associated with those budgets. The chapter was used to give 
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some background into the current rejuvenation performance 
measurement and budgeting have found in the Government 
Performance and Results Act. It should be noted that all 
past attempts at instituting performance budgeting have 
failed on a full scale level. While GPRA does not mandate 
performance budgets, it is a step in that direction. From 
its unanimous votes in both the House and Senate it appears 
to be here for the near future. Thus federal agencies need 
to prepare for the inevitable performance issues soon to be 
mandatory. On the other hand, history has shown that it is 
extremely difficult to enact changes in the current 
budgetary process. The next chapter will show the 
difficulty associated with such attempts. 
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Example of program and performance budgeting for highway 
maintenance. 










Figure 3: 
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III. PERFORMANCE MEASUREMENT AND BUDGET REFORM 


The Government Performance and Results Act of 1993 
requires all federal agencies to submit a performance plan 
by September of 1997. This is to be included in the FY99 
budget submission for the agencies.(P.L. 103-62, 1993) The 
apparent long term goal for GPRA is to implement performance 
budgeting as a means of resource allocation for federal 
agencies. This is readily identified by the performance 
budget pilot projects which are to be conducted in FY98 and 
FY99. Because the "final product" of the GPRA is presumed 
to be performance budgeting, this thesis will now give a 
brief synopsis of the budgetary reforms of the past century. 
The common thread between the reforms included is the 
attempt to raise the level of efficiency and effectiveness 
by which the government operates via performance measurement 
of some sort. 

In his book "Budgeting for Modern Government", Donald 
Axelrod identifies eleven major budget reforms which have 
occurred since the turn of the century. (Axelrod, 1988, pg. 
259) These reforms consisted of: 

• Executive Budgets 

• Functional Budgets 

• Program and Performance Budgets 

• Multi-year Expenditure Projections 

• Unified or Comprehensive Budgets 

• PPBS (Planning, Programming Budget System) 

• MBO (Management by Objective) 

• Productivity Budgets 

• ZBB (Zero Based Budgets) 
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• Budgets as Tools for Economic Management 

• Legislative Budgets 

Figure 4 is a time-line which shows the approximate 
starting date for the major reforms discussed in this 
thesis. This is the year in which the reform started 
receiving considerable consideration or was in fact adopted. 
Some of the more important budget legislation enacted and 
the requirements of GPRA are also included. 


Date _ Event 

1921- - Executive Budget; Budget and Accounting Act 


1948- - Functional Budget 

1949- -Performance/Program Budgets; Budget/Accounting 
Procedures Act 


1961- - PPBS, DoD 

1965- - PPBS, all federal 


1971- 

1974- 

1977- 

1980- 


Management By Objective (MBO) 
Executive, Congressional Budget Act 
Zero-Based Budget (ZBB) 

Budget for Economic Management 


1993- 

1994- 

1995- 


Government Performance and Results Act 
Performance Plan pilots commence 
Managerial Accountability pilots commence 


1998- - Performance Budget pilots commence 
2001-^ - Decision on performance budget pilots 


Figure 4 : Budget Reform Time-line 
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The most enduring budget reform is the executive budget 
instituted by the Budget and Accounting Act of 1921. This 
was an attempt to consolidate the previously fragmented 
budget process into a cohesive system as developed by the 
chief executive. (Axelrod, 1988, pg. 260) Prior to this 
act, the various committees in Congress with oversight of a 
particular program would entertain budgets from each federal 
government agency separately via the Treasury Department. 
These budgets were handed out in lump sums with little or no 
executive branch attention. 

The Budget and Accounting Act of 1921 specifically 
tasked the President to submit a budget, including estimates 
of expenditures, appropriations, and receipts for the 
ensuing year. The Bureau of the Budget (BOB) was also 
created in the Treasury Department. (Lynch, 1995, pp. 40-42) 
A line-item of expense format was used as the basic building 
blocks for the executive budget. This required each agency 
or responsibility center to arrange its budgets into 
categories such as salaries, insurance, office supplies, 
medical expenses,etc. These are the items required to run 
the government process and the sum of money needed annually 
to purchase them is thus identified. 

The idea of having the chief executive responsible for 
budgeting seems like a tremendous one. However, the format 
used for arranging the budget is not necessarily the best 
one available in the eyes of many observers. From the 
moment of its inception, executive budgeting came under 
intense criticism. The critique of this budget process was 
summed up by Lent D. Upson as follows: 

Jt focused on inputs (money, staff, materials) 
instead of outputs or results. No program 
objective was used as a goal and therefore what 
was to be accomplished was not defined. (Upson, 

1924, pg. 73) 
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still other reformers noted that the budget format 
focused on marginal changes and not on periodic review of 
efficiency and effectiveness of governmental programs 
(Axelrod, 1988, pg. 261). These criticisms led to a search 
for a budgeting system which could address these issues. 

This search is still in progress today. 

Functional budgets were the first attempt to rectify 
the shortcomings of the executive system. They are an 
attempt to group expenditures by major function or broad 
purposes vice object of expenditure. These functions were 
to be arranged without regard to responsibility center or 
object of expense. This format has been used within the 
federal government for many years but did not become part of 
the "official" federal budget process until 1974. The 
Congressional Budget Act of 1974 made functional budget 
classifications law. (P.L. 93-344, 1974) As a result of 
this act the federal budget was broken down into 17 major 
functional categories and 4 "other' categories of 
expenditure. The functional areas are then broken down into 
subfunctions and individual programs based upon the missions 
of the major category. This was a good start towards 
performance concerns because it identified the primary 
functions of a government entity; however, functional 
budgeting did not emphasize the need for efficiency 
measurement within a particular function. 

The third major budget reform discussed, program and 
performance budgets, stems from the inadequacies of the 
first two. The popularity of these budget styles grew from 
the Commission on Organization of the Executive Branch of 
the Government (commonly referred to as the Hoover 
Commission) in 1949. The Commission concluded that if the 
federal budget were prepared on a performance basis, 
centering the attention on the amount of work to be 
achieved, and cost of this work "...Congressional action and 



executive direction on the scope and magnitude of the 
different federal activities could then be appropriately 
emphasized and compared for resource allocation (Hoover 
Commission, 1949). And more importantly, the cost and 
achievements of the federal government would be furnished to 
the Congress and the people. 

Performance budgeting was initially mandated by 
amendments to the National Security Act in 1949. These 
amendments required DoD to install performance budgeting in 
the three services (63 Stat 578, 1949). The federal 
government as a whole entered into performance budgeting as 
a consequence of the Budget and Accounting Procedures Act of 
1950. This act required the heads of each agency to support 
"budget justifications by information on performance and 
program cost by organizational unit." (64 Stat 832, 1950) 

At the same time performance budgeting began to spread 
throughout local and state governments. Early attempts 
included Detroit, Mi., Kissimmee, FI., San Diego, Ca., the 
states of Oklahoma, California, and Maryland, as well as 
many others (Seckler-Hudson, 1953, pp. 5-9). To this day, 
several states and cities still practice some form of 
performance budgeting. 

The U.S. was not alone in its recognition of the 
beneficial aspects of performance budgeting. Nearly fifty 
countries implemented various aspects of performance and/or 
program budgets in the I960's. Among the leaders in this 
endeavor were Sweden, Britain, Canada, and France. Most 
attempts in foreign nations merely supplemented the 
traditional budget and were usually issued as separate 
documents altogether. (Axelrod, 1988, pp. 272-3) Whether or 
not any major allocation of funds is affected by the 
performance budgets is questionable. These same problems 
were mirrored in the U.S. federal budget process as well. 
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Program and performance budgeting were to be an 
integral part of the Planning, Programming and Budgeting 
System (PPBS). The first agency to implement PPBS was the 
DoD in 1961 under then Secretary of Defense, Robert 
McNamara. PPBS was the most widespread reform movement 
since the executive budget. According to John Wanat, PPBS 
"is an attempt to use budget preparation as an occasion to 
evaluate rationally the programs an agency engages in so as 
to choose the programs most appropriate to the agencies' 
goals." (Wanat, 1978, pp. 98-99) The system was backed up 
by analytical tools such as cost benefit analysis, systems 
analysis, and cost effectiveness analysis. This system was 
designed to provide greater rationality and efficiency in 
the allocation of resources. 

Again, local, state and foreign governments jumped on 
the bandwagon of budgetary reform along with the federal 
government. After all, this reform had combined the best 
parts of all the previous reforms together into one large 
systematic package. In 1965, President Johnson announced 
the institution of PPBS as the budgetary system to be used 
throughout all federal agencies. However, PPBS proved to be 
far too cumbersome to be adapted on that grand a scale for 
most of the agencies involved. In fact the only remnants of 
this system lie with the agency that first introduced it, 
that being the DoD. Just four years after its full 
implementation. President Nixon killed PPBS as the federal 
medium for budgetary creation (Axelrod, 1988, pg. 287). 

Management by Objective (MBO) was the Nixon 
administration's answer to the difficulties associated with 
the PPBS system. The focus of MBO was on selective 
objectives and their associated costs, along with controls 
to measure performance as execution of the budget occurred. 
This was not so much a system as it was an approach. For 
the Nixon administration it was a way of getting a handle on 
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the goals of a particular program. (Axelrod, 1988, pg. 294) 
Again an attempt had been made to shift the emphasis from 
input control to a results orientated approach based on 
performance measurement of specific goals. Unfortunately, 
the Nixon administration was beleaguered with other more 
pressing problems, and as a conseguence MBO fell by the 
wayside as a budget instrument for the federal government. 
The end of the Nixon reign for all intents and purposes 
ended the MBO approach as a viable budget candidate. 

The Carter administration also brought with it a reform 
for the budget process. Zero Based Budgeting (ZBB) was 
adopted to shift the focus of budgeting from sole concern 
with new and expanding programs to a zero based review of 
all programs on an annual basis. (Taylor, 1977, pp. 33-34) 
Again performance measurement played a vital role within the 
proposed reforms. When the Office of Management and Budget 
(0MB) divulged its guidelines for ZBB implementation, it 
required agencies to review several aspects of each program, 
including the following (Axelrod, 1988, pg. 296): 

• Objectives, which should be explicit statements of 
output 

• Performance measures to gauge efficiency, 
effectiveness and workload of the decision unit 

• Actual measurement of accomplishments 

• Resource requirements and program information 

For various reasons, ZBB has taken the path of most of 
its predecessors. From excessive processing costs to its 
inability to allocate resources anymore than at the margin, 
ZBB had many criticisms made in its behalf. ZBB was perhaps 
the last of the major budget reform attempts until 
performance budgeting once again surfaced in the early 
1990's. 
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What have been the effects of the past one hundred 
years of budget reform? While none of the major reforms 
exist as a whole, many pieces of each exist today throughout 
local, state and the federal government. Expenditure 
control, planning and cost evaluation, performance 
measurement, multi-year budgets, cost-benefit analysis, 
unfunded requirement review, unit costing and systems 
analysis are all in some way related to previous reform 
attempts. It should be noted that this is in no way an all 
inclusive list; many other legacies of reforms exist today 
as well. Yet, there is one acid test of budgetary reform 
which none of the past attempts have been able to pass since 
the advent of the executive budget. Allen Schick puts it 
best: 

...did the innovation alter the basis for making 
budget decisions? Only if the answer is "yes" can 
an innovation be considered successful. (Schick, 

1982, pg. 91) 

Since the primary base for resource allocation is still 
the object-of-expense, it appears Schick would say budget 
reforms to date have failed. Moreover, other experts in 
budgeting felt that because of the political nature of 
budgeting, several types of budget reform are doomed to 
failure from the start. Aaron Wildavsky claims that 
incrementalism is the only true budgetary process capable of 
success, since to do otherwise invites large-scale political 
warfare (Wildavsky, 1978, pg. 6). Whether the current call 
for performance budgeting will have any long-term effects 
remains to be seen. 

This history has spent considerable time discussing 
reforms other than just performance budgeting of the 1950's. 
This was purposeful in that the author believes most budget 
reforms have been initiated in order to improve the 
government's programs with respect to efficient and 
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effective use of resources. In all the reforms presented, 
performance measurement played a significant role in the 
improvement attempts. As indicated by the past one hundred 
years of budget reform, performance concerns are not new. 
However, the reforms of the past have not been able to make 
performance measurement a major player within the budgetary 
process. To date, the ever popular "object-of-expense" 
still retains its allure to the reviewers of the federal 
budget. The Government Performance and Results Act will 
once again try to show the benefits of using performance 
measures as a resource allocation tool. 

The next chapter will turn to the Department of 
Defense's pilot project initiatives. These pilots are a 
result of the GPRA and are being used as the stepping stone 
DoD will need to move into full implementation of 
performance measurement as required in FY99. 




IV. DOD IMPLEMENTATION OF GPRA 


A. DOD CORPORATE LEVEL 

The GPRA will require all federal agencies to submit a 
performance plan for FY99 in the September 1997 budget 
submission. This plan is to include the items described in 
Table 2. While these requirements deal strictly with 
performance measurement, they provide a bridge to the 
ultimate objective of GPRA, that being performance 
budgeting. 


Table 2: GPRA Performance Measure Rec[uirements 


p 


B 

Express such goals in an objective, quantifiable, and 

measurable form 

■ 

Briefly describe the operational processes, skills and 
technology, and the human, capital, information, or other 
resources required to meet the performance goals 

■ 

Establish performance indicators to be used in measuring or 
assessing the relevant outputs, service levels, and outcomes of 
each program activity 

B 

Provide a basis for comparing actual program results with the 

established performance goals 

B 

Describe the means to be used to verify and validate measured 

Source: P.L. 103-62, 1993. 

According to the Office of Management and Budget (OMB), 


there are 75 pilot projects located in 26 major federal 
agencies currently involved in the performance plan/report 
phase of GPRA. (Hamre, 1995) This accounts for over 20 
percent of the non-postal federal work force, a rather 
sizeable effort for a pilot project. As a major federal 
agency, the DoD was selected by OMB to participate in this 
phase as well. 















The DoD has a notable interest for involving itself in 
the GPRA pilot phases given its current budget status. From 
its peak in T'iZS, defense authorization had declined by 34.3 
percent to its current FY95 level. Outlays for defense have 
declined by 25.9 percent over the same time period. (Berner 
and Graney, 1994, pg. 18) With the end of the Cold War, 
public sentiment no longer will tolerate what are perceived 
to be overly-generous defense budgets. However, the DoD has 
not experienced the corresponding decline in operating tempo 
one would expect with the Cold War demise. For DoD, this 
means doing more or doing the same with less money. The 
GPRA proposes to do just that for federal agencies by 
raising the level of efficiency and effectiveness by which 
they operate. Thus it has become imperative that the DoD 
take a serious look at how the GPRA can help it survive with 
budgetary constraints that do not appear to be going away 
soon. 

The Secretary of Defense has designated the Under 
Secretary of Defense, Comptroller, USD(C), to lead its GPRA 
implementation strategy. In doing so, the USD(C) has 
decided to approach the GPRA from both a corporate DoD level 
as well as an agency perspective. The pilot projects have 
been conducted at agency levels vice the upper corporate 
layer. At the corporate level, the USD(C), along with 
representatives from all the other Under Secretary and 
Assistant Secretary Offices as well as the Joint Chiefs of 
Staff, have formed a working group to implement GPRA. 

This group was tasked by the Secretary of Defense with 
developing the strategic mission statement, a DoD vision 
statement and DoD-wide corporate goals. (Hamre, 1995, pg. 1) 
Figure 5 represents the draft version of their output to 
date. The intention is to include these statements in the 
next Defense Planning Guidance. While some minor changes 
may still take place, the group has reached a general 
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consensus on the content of these statements. These 
statements are a start towards fulfilling the first major 
requirement of GPRA full implementation, that being to 
create a strategic plan. 

The next step for the group is to produce the draft 
strategic plan. This is to be completed by 1 October 95. 
(Hamre, 1995, pg. I-l) This will allow the DoD to capture 
performance data during FY96. Thus, DoD will have 2 years 
of experience at performance planning and reporting at the 
corporate level prior to the legally mandated September 1997 
submission data. The group is now engaged in developing 
general performance goals for each of the seven corporate 
level goals. Lastly, the group is looking into providing 
guidance in the POM (Program Objective Memorandum) 
Preparation Instructions for Fy97-01 regarding corporate 
goals and performance measures. These will act as the 
guides for the agencies within DoD in providing performance 
data to the corporate level. 

These are the issues currently being worked on at the 
upper levels of the DoD. However, the pilot projects have 
been delegated down to the agency levels of the DoD. This 
seems like a reasonable strategy for pilot implementation 
given the tremendous size of the DoD compared to most other 
federal agencies. The next section will analyze the pilot 
projects under the auspices of GPRA in progress within DoD. 

B. PILOT PROJECT COMPARISON 

In order to decide which commands would participate in 
the pilot projects for performance plans/reports, volunteers 
were solicited by the DoD. The volunteers had to be 
nominated by the Secretary of Defense, then final approval 
was required by the Director, Office of Management and 
Budget. (OMB, 1995) The pilots nominated and selected are 
the Defense Logistics Agency (DLA), Defense Commissary 
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Agency (DeCA), Air Combat Command (ACC), Army Research 
Laboratory (ARL), Commander in Chief, U.S. Atlantic Fleet 
(CINCLANTFLT) Carrier Battle Group and the Department of the 
Army, U.S. Army Corps of Engineers Civil Works Operation. 

The selection of these seven agencies/commands appears 
to be a good mix. To start with, these commands and agencies 
are all volunteers in this endeavor. Using volunteers vice 
designees ensures a commitment by the various groups to 
make GPRA work. Of the commands selected, two are major 
combatant commands, one is a research facility and the 
others can be classified as service oriented agencies. This 
diversification in types of commands will be beneficial in 
determining what types of agencies will be conducive to 
performance measurement and what types will have problems 
with these measures. One might speculate that it will be 
easier for the service agencies than the research or 
operational commands to develop performance plans/reports. 
The difficulty with performance measurement in the research 
and operational cases is trying to quantify their outcomes. 

The first agency selected as a pilot project within the 
DoD was the Defense Logistics Agency (DLA). Chosen as a 
pilot for all three years covered in the performance 
plan/report phase, DLA has already submitted performance 
plans for Fy94 and Fy95. DLA has also submitted the first 
performance report, for Fy94, from the DoD. DLA describes 
itself as a Combat Support Agency, responsible for providing 
the Military Services with a broad range of logistics 
support. (Defense Logistics Agency, 1994, pg. 1) DLA 
employs over 58,000 civilian and military personnel. Its 
facilities include supply centers, distribution depots, 
property disposal offices, contract administration offices, 
and contractor in-plant residences. Annual sales of over 
$11 billion, distribution of $102 billion worth of material 
and contract administration for contracts with a face value 



of over $840 billion make DLA a formidable pilot project for 
the GPRA. The process used by DLA in its performance 
planning and reporting is the subject of chapter four and 
will be discussed in detail at that point. 

The first operational command selected as a pilot was 
the Air Force's Air Combat Command. ACC is the major 
combatant command of the Air Force with cognizance over 
theater, nuclear and air defense forces. Unlike most of the 
other pilots, ACC has taken a "non-corporate" vantage for 
developing their performance plans. Three operational air 
wings have been selected under ACC to develop their own 
performance plans in lieu of a corporate ACC plan. (Air 
Combat Command, 1994, pg. 4) These three plans were then 
simply collated into a single document labeled the ACC 
performance plan. These wings make up 13,500 personnel and 
have a $120 million annual budget. ACC is implementing the 
GPRA criteria via its "Quality Air Force" initiatives 
currently in progress throughout the Air Force. This seems 
like a reasonable endeavor since the concepts of Total 
Quality Leadership and performance measurement have 
ideological similarities. ACC has submitted a plan for FVr95 
and will also develop one for Fy96. 

The Defense Commissary Agency (DeCA) has recently 
undergone a consolidation process and wanted to use the GPRA 
pilot as a means for driving home strategic planning and 
process improvement. DeCA's mission statement is "To 
operate the most efficient and cost effective commissary 
system as possible". (Defense Commissary Agency, 1994, pp. 
i-2) DeCA has approximately 20,000 employees, 350 world wide 
sites, and sales of about $5.9 billion. Like the ACC, DeCA 
has submitted a performance plan for FY95 and will also 
submit one for FY96. 

The Army Research Laboratory (ARL) was an interesting 
choice as a pilot project. The prospects of trying to 
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measure the output, much less outcomes, of a research 
facility are daunting. The ARL is the central research 
laboratory of the Army Material Command and employs 
approximately 3600 people, including 1800 scientists and 
engineers. (U.S. Army Research Laboratory, 1994, pp. 1-4) 

ARL is using its nomination as a pilot to enhance its three 
primary management initiatives of creating a Federated 
Laboratory, business process re-engineering and laboratory 
construction under the BRAC initiative. These initiatives 
are designed to produce out-sourcing in research, 
downsizing, and improving quality. As with the previous 
pilots, ARL has already submitted its first performance plan 
and will submit its second for Fy96. 

The U.S. Army Corps of Engineers Civil Works Operation 
is responsible for managing the water resources infra¬ 
structure that provides for navigation, flood control, 
hydropower, recreation and natural resources throughout the 
U.S. and represents 1,400 projects in all. (U.S. Army Corps, 
1994, pp. v-1) The Corps is using the GPRA pilot to help 
enhance their O&M Program Improvement Plan. This enterprise 
is trying to provide a justified (effective) level of 
service in the least cost (efficient) manner for the Corps. 
The O&M funding level for the Corps is about $1.7 billion 
with a work force of 14,000 individuals. The Corps has 
submitted their first performance plan and will also submit 
one for FY96. 

The last two pilots were officially approved by 0MB in 
January of 1995. The Army Audit Agency (AAA) and a 
CINCLANTFLT Carrier Battle Group made up the final DoD 
pilots. The AAA is the centralized internal audit 
organization of the Department of the Army. The function of 
the AAA is to assist the Secretary of the Army in satisfying 
statutory and fiduciary responsibilities as well as 
assisting line managers in making informed decisions. 
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resolving issues and using resources effectively. (Army 
Audit Agency, 1995, pp. 1-3) The AAA has about 700 
employees, including 600 professional auditors. Their 
interest in becoming a pilot stemmed from the Total Quality 
Management philosophy they adopted in 1993. Assuming their 
nomination would be approved, AAA submitted a performance 
plan for Fy95 and will also provide a FY96 plan. 

U.S. CINCLANTFLT was chosen as the second operational 
command for the GPRA pilot project within the DoD. 
CINCLANTFLT is the Navy Component Commander of the Unified 
CINC's of the Atlantic, Strategic and Southern Commands and 
the Navy's major force provider to Europe, Central and 
Special Operations Commands. (Commander In Chief, U.S. 
Atlantic Fleet, 1994, pp. 1-3) They have decided to use a 
Carrier Battle Group as the platform for their pilot 
project. A Group consists of approximately 11 ships, one 
air wing and over 7,800 military members on board. The 
resources required to operate a Group over its entire 
workup/deployment schedule is about $274 million. This 
cycle takes approximately 18 months to compete. The 
attractiveness of using a Carrier Battle Group is that it 
contains representatives of all the major combatant forces 
available to the Atlantic Fleet. This pilot will help 
demonstrate the ability to measure the performance of 
equipping, training, and operating a major combatant force. 
Because of the difficulty with developing performance 
measures for the Group, CINCLANTFLT will only participate in 
the project for FY96. 

The preceding paragraphs have briefly described the 
pilot project agencies and commands. Next the thesis will 
take a look at the performance plans submitted by the pilots 
in a comparative analysis format. The definitions of the 
performance measures given in the third chapter will be used 
in comparing the measures used by the pilots. 
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Table 3 is a synopsis of the analysis. The first 
column indicates the command or agency engaged in the pilot 
project. The second column indicates the primary 
orientation of the command, (e.g., is the pilot service 
oriented, or operational, etc?) The third and fourth 
columns indicate the level of resources used by the agency 
in the form of budget and personnel. (The resources 
indicated for ACC are only those used by the three air-wings 
engaged in the pilot process, not the resources available to 
the entire ACC command.) Column five shows the total number 
of measures included within the pilot's first performance 
plan. (The DLA FY94 plan contained several other measures; 
however, these measures were indicated as future ones and no 
targets/goals had been set for them. Thus, they were not 
included in this analysis.) 

The last five columns of Table 3 represent the five 
types of performance measures as described in chapter III. 
They are arranged on a spectrum from least difficult to 
capture (input) to the most difficult (outcome). Indicated 
for each pilot is the number of each measure included in its 
plan and the percentage this is of the total number of 
measures in its plan. (Percentages may not add up to 100% 
since they were rounded to the nearest whole percent.) For 
example, ARL has included six output measures in its FY95 
plan. This makes up approximately 32 percent of the total 
measures in the plan. Some examples of each type of measure 
are provided in Figure 6 for clarity. 
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Input Output 

-Appropriated $ (DeCA) -Disposal Sales Proceeds (DLA) 
-No. of Employees on -No. of NRC Advisors (ARL) 

long-term training (ARL) 

Efficiency 

-Unit Cost per Barrel 
Fuels (DLA) 

Eng) 

-Medical Wor)c Unit Cost 
(ACC) 

Outcome 

-Maintenance Metric Index* (ACC) 

*This measure can only be considered an outcome from the 
maintenance manager's vantage, not from ACC as a whole. 


Effectiveness 

-Customer Satisfaction (ARL) 
-Unit Availability (Corp of 


Figure 6: Measurement Examples 

Some rather interesting results can be gleaned from 
Table 3. First, a comparison of the agency size with the 
number of measures used might be useful. By far, the 
largest pilot as measured by resources used is DLA. It is 
approximately three times the size of the nearest pilot in 
both budget and personnel. Twenty-two measures for an 
agency this size does not seem unreasonable to the author. 
However, DLA does not have the largest plan with regard to 
number of measures. ACC tops the list with thirty-two 
measures in all. The budget authority covered by 
these thirty-two measures is less than one-tenth that of 
DLA's. ARL also has a large measure/ resource ratio as 
compared to DLA. With nineteen measures, just short of 
DLA's twenty-two, ARL is measuring the performance of 
resources with a value of about 5 percent of DLA's. In 
fact, DLA has the lowest measure to resource ratio of all 
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the plans. (The author must note that there is no magical 
measure/ resource ratio value, this is simply used as a 
point of comparison.) Overall, the trend appears to be 
toward more rather than fewer numbers of performance 
measures. 

Another interesting result taken from Table 3 is how 
different types of agencies chose measures on the spectrum 
of those available. The service type commands tended to 
choose measures more evenly distributed across the entire 
spectrum of measures. This contrasts with the research and 
operational commands. The research command, ARL, included 
84 percent of its measures in the input/output categories. 
The operational command shifted to the opposite end of the 
spectrum in that 72 percent of its measures were 
effectiveness and efficiency indicators. The one exception 
to this generality was the Army Corp of Engineers which used 
four of six measures of effectiveness. As can be seen by 
the bottom row of Table 3, the plans as a whole spread 
across the spectrum rather evenly. Input, output and 
efficiency measures all have about 20 percent of the total. 
Effectiveness measures are used approximately twice as often 
as the others. The one glaring exception is the lack of 
outcome measures provided. Despite the shortage of outcome 
measures, the plans consisted of almost 60 percent higher 
order measures (i.e., efficiency and effectiveness). 

Some other results of the analysis conducted in 
conjunction with this thesis are not readily apparent from 
Table 3. These results address the following: 

• size of the overall plan itself 

• complexity of the measures 

• how the measures are displayed 

• relation of the measures to agency goals 
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• correlation with the budget 

• ease of capturing the measures 

Plan Size; A couple of plans appeared to be 
overwhelming in size. The initial DLA plan was one hundred 
and twenty pages long. Only forty of those pages dealt 
directly with the FY94 performance plan itself. The rest of 
the plan was used to describe the thirty-plus strategic 
initiatives currently in progress at DLA. Another plan 
which spent considerable time describing items not directly 
related to the performance plan was the FY95 ARL plan. 

Here, twenty-two pages were spent describing the "Technical 
Objectives" upon which management is focused. Again these 
objectives are only related to the performance measures 
indirectly. Only eleven pages were used to discuss the 
actual performance plan. Dis- counting DLA's first plan, 
average length is about 30 pages. 

Plan Complexity: A few plans used performance measures 
which were far too complex for the average plan reviewer to 
understand. They were measures that only an insider would 
understand. An example of this is ACC's "Maintenance Metric 
Index". ACC defines this measure as: 

...a summary of the maintenance effort required to 
support the operational squadrons and training 
squadrons of the 3140G. This summary is weighted. 

The sub elements are: mission capable, launch 
reliability, aircraft scheduling effectiveness, 
maintenance scheduling effectiveness, maintenance 
delivery reliability, 12 hour fix rate, break 
rate, combined abort rate, cann rate, delayed 
discrepancies, and repeat/recur average... (ACC, 

1994, pg. 17) 

No less than twenty-two calculations are required to 
complete the measure. The eleven elements are weighted and 
then totalled to give a final percentage. While the eleven 
elements are described in the plan, several of these 
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elements may still not be understood by individuals outside 
the air community. Moreover, the final percentage is given 
as 94.7 percent. No targets are given and no baseline data 
is supplied either. Thus, an administrator would have a 
hard time gauging the performance associated with this 
measure and difficulty in deciding how much performance 94.7 
percent will buy them. 

Plan Arrangement; Several of the agencies found ways 
to display its measures in an easily understood manner. 

These particular plans also tended to allow the best 
evaluation possibilities. DLA and ARL seemed to have 
displays which were exceptional. An excerpt from the ARL 
FY95 plan shows this arrangement (ARL, 1994, pg. 24). 

Metric: Actual Goal Actual 

FY93 FY94 FY94 

No. of invention 

disclosures 166 100 84* 

(* YTD as of 31 May 94) 

As can be seen, ARL expects their wor)cload for 
Invention Disclosures to decline from its FY93 level. The 
baseline shown in FY93 is considerably higher than the goals 
set for the next several years. Also indicated is the fact 
they have completed 84 percent of this year's goal eight 
months into the fiscal year. This type of display would 
allow administrators to as)c questions such as: 

• Why was the goal reduced? 

• Were the FY93 results just a fluke? 

• Is downsizing causing the decline in workload? 

• Is ARL phasing out this type of work? 

Thus, trends can be observed from one year to another. With 
a breakdown of this fashion, administrators can ask 


Goal Long-term 
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intelligent questions about the program during their 
assessment. 

Goal Relation; Similar to the previous analysis 
result, some agencies seem more adept at coordinating 
performance measures with the overall goals of the agency. 
DeCA, DLA, ACC and ARL all were able to link the indicators 
directly to goals in an efficient manner. For example, 

DeCA arranged its plan such that the goal and measure were 
identified together; an excerpt is provided below (DeCA, 
1994, pg. 5). 

GOAL: MAXIMIZE CUSTOMER SATISFACTION 

Objective: Improve customer service at the commissary level . 
Performance Indicator: Customer Service Evaluation System. 
Performance Goal: Annual increase in CSES. 

Baseline: Fy94-average CSES score is 86%. 

This allows the administrator to identify the overall 
corporate goal to which the performance indicator is most 
closely associated. 

Budget Correlation: Correlation with the budget is 
also a considerable concern when it comes to these plans. 

In all cases the plans identified the particular accounts 
from which it receives funds. For example, ACC indicates 
that it receives funds from the following accounts (ACC, 
1994, pp. 8-9): 

57-3500-0-1-051 (Military Personnel, A.F., partial) 
57-3400-0-1-051 (Operations&Maintenance, A.F., partial) 
57-3080-0-1-051 (Other Procurement, A.F., partial) 

ACC receives only a portion of each fund. The activities of 
ACC are unique from any other command in the Air Force. 
However, the funds for these other commands also come from 
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the same accounts. Consequently, the question then becomes 
how administrators decide which agency gets how much 
funding. GPRA would base this upon the performance of the 
activities within the agency. Thus a disconnect exists 
between the activity performance and the funds which can be 
allocated to the activity. ACC recognizes this problem 
within its FY95 plan when it states: 

While ACC has made significant strides in 
performance measurement, strategic planning, and 
integrating the Quality Air Force program into its 
daily operations; it at present, has limited 
ability to tie these programs to the budget or to 
derive associated cost per unit of output 
measures. (ACC, 1994, pg. 8) 

Ease of Capturing Measures: The last attribute looked 
at during the analysis of the performance plans was the ease 
of capturing the measures used by the agencies. Some plans 
included measures which would be inherently unmeasurable. 
These type of measures have goals which simply state 
"reduce", "minimize", or "develop" some aspect of a program. 

A prime example of this is the Army Corp of Engineers 
"Industry delay cost due to unscheduled closures" 
measurement. The goal associated with this measure is to 
minimize the cost to the navigation industry resulting from 
unscheduled lock closures. (Corp of Engineers, 1994, pg. 5) 
No targets or baseline data were provided beyond this. 

Several plans took this type of measure only one step 
farther in that they attached a numerical goal to the 
measure such as "increase by 10%". One such example comes 
from the FV95 DeCA plan. The performance indicator in 
question was the "DeCA regional work force diversity". The 
goal associated with the measure was to simply "get a 2% 
increase in categories which have an imbalance". 



To review this chapter, the strengths and weaknesses of 
the plans will be summarized. First of all the strengths 
are presented. 

First Attempts at Performance Planning: As the first 
agencies to produce performance plans, these pilots will 
help future agencies with the creation of plans when they 
become law. 

Goal Linkage; Several plans were able to directly 
relate performance measures with overall strategic goals. 
This is extremely important since the efficient and 
effective accomplishment of the primary goals of an 
organization is what the writers of GPRA desired. 

Measurement of Outputs. Efficiency and Effectiveness: 

As indicated by Table 3, the plans as a whole used 
approximately 80 percent output type measures. (Efficiency 
and effectiveness measures are outputs adjusted for cost or 
compared to pre-set standards.) GPRA specifically calls for 
these types of measures under section 1115. 

...establish performance indicators to be used in 
measuring or assessing the relevant outputs, 
service levels and outcomes of each program 
activity. (P.L. 103-62, 1993) 

Taraet/Goal Indentification; A few agencies found ways 
to easily articulate the goals and targets for performance 
indicators. Providing baseline data, targets for the 
current year and future years allows the administrators to 
see trends in the program instead of just raw current year 
workload numbers. This also provides a means for asking 
intelligent questions about the program's activities and the 
associated performance. 

Recognition of Budget Dilemma; Finally, these plans 
have shown how difficult a task it is to tie performance 
measures directly to the budget. As long as the budget 
continues to use object-of-expense as its primary basis. 
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performance measurement will have little effect on resource 
allocation at the federal level. 

Several weaknesses in the plans were also noted during 
this analysis. 

Bulk of Plans: Several plans were extremely large and 

discussed items not directly related to the performance 

measures contained in the plan. When GPRA-mandated plans 
are to be submitted in 1997, if the individual agency plans 

are as large in sheer volume as some of the pilots, the DoD 

will be swamped with data in preparing its corporate plan. 

To take that one step farther, 0MB would seem to face a 
daunting task if all federal agencies were to supply such 
extensive performance plans. 

Number of Measures: Related to the above problem is 
the number of plans which contained a large performance 
measure to resource ratio. In the DLA Fy95 plan (to be 
discussed in more detail next chapter), the ten activities 
in which DLA engages are covered by only 24 measures. An 
average of 2.4 measures per primary activity seems 
appropriate. 

Baseline/Taraets Missing: A few plans did not contain 
specific target values for their measures. Several which 
did supply targets, failed to provide baseline data as a 
point of comparison. One of the obvious reasons for this 
problem is the fact that in many cases this is the first 
performance measurement occurring within the agency and 
therefore no previous data exists. 

Lack of Outcome Measures: None of the plans contained 
"true" outcome measures. This can be attributed to the fact 
that outcomes are extremely difficult to define much less 
capture. This may also result from the author's broader 
view of outcome measures than those apparent in the pilot 
plans. Outcome measures should measure the impact of a 
program on the customers of that program. Many outcomes 
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cannot be measured annually and thus may be difficult to 
include in a GPRA plan. 

Too Many Input Measures: Finally, some agencies relied 
heavily on input or internal measures as a means to gauge 
the performance of their activities. These types of 
measures are not required by GPRA. Moreover, some agencies 
used measures which only an insider would understand. 
Agencies need to keep in mind that the whole purpose for 
these plans is to report performance to external agencies. 
Internal and input measures are probably of little concern 
to external stakeholders. 

This chapter described the steps currently being taken 
by the corporate level within DoD in its preparations for 
GPRA requirements to come. It also analyzed the seven 
performance plans which have been submitted by the various 
DoD agencies involved in GPRA pilot projects. All these 
agencies are volunteers in this process, and are therefore 
on the cutting edge of performance reporting for the DoD. 

The purpose of the analysis contained in the chapter is to 
identify problems associated with performance measurement 
and reporting in conjunction with GPRA implementation. 
Several features of plans were presented as examples for 
those agencies which will soon be required to create plans 
for the DoD. Also provided were some suggestions by the 
author as to how the plans might be presented under GPRA. 

The next chapter will conduct an in-depth analysis of the 
process used by the Defense Logistics Agency in its 
implementation of the Government Performance and Results 
Act. 
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V. DLA PILOT ANALYSIS 


A. INTRODDCTION TO DLA 

The Defense Logistics Agency was the first chosen 
within DoD to act as a pilot under the auspices of the GPRA. 
DLA is the logistics division of the DoD and provides 
material and logistical services to all the military 
services. Figure 7 presents the strategic mission statement 
and vision statement as indicated by the DLA Corporate Plan. 


DLA Mission: The Defense Logistics Agency is a combat 
support agency responsible for worldwide logistics 
support throughout the Department of Defense. The 
primary focus of the Agency is to support the warfighter 
in time of war and in peace, and to provide relief 
efforts during times of national emergency. 

DLA Vision: To be the provider of choice, around the 
clock - around the world... providing the logistics 
readiness and enabling weapon systems acquisition at 
reduced cost... by leveraging our corporate resources 
against global logistics targets... and finding savings 
through teams, improved business practices, and 
technological breakthroughs. 


Figure 7: DLA Mission and Vision Statements 

Source: The DLA Corporate Plan, 1994. 

Not only is DLA the first pilot project within DoD, but 
it is also by far the largest. Figure 8 is a compilation of 
DLA's financial statistics which shows the annual budget 
authority, number of employees, and volume of the various 
businesses in which DLA engages. These statistics make DLA 
larger than the next closest pilot project within the DoD by 
at least three-fold. 
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Annual pudqet 
$14.6 billion 


Employment 

-58,000 


Annual Bales 
$11.6 billion 


$9.4 billion 


Contract Mat. 
$842 billion 


Figure 8: DLA Financial Statistics 

Source: DLA Corporate Strategic Planning Office, 1995 

The agency is currently headed by a Vice Admiral of the 
U.S. Navy. The organizational structure of DLA is broken 
down into three major business areas; Supply Management, 
Distribution, and Contract Management. Figure 9 is an 
abbreviated version of the DLA organizational chart, showing 
the major offices associated with the GPRA implementation 
efforts. The Supply Management and Distribution functions 
are controlled by a Material Management Deputy Director. 

The Contract Management function is headed by an Acquisition 
Deputy Director. The final Deputy Director is in charge of 
the Corporate Administration Division. 

The office charged with GPRA implementation is the 
Office of the Executive Director, Strategic Programming and 
Contingency Operations. This office is located within the 
Corporate Administration division. The performance plans 
created for the GPRA are developed within the Planning 
section of the Corporate Strategic Programming Office 
located within the Executive Directors office. 

DLA began its pilot project on 22 October 1993 when 
they sent their nomination request to the Office of the 
Secretary of Defense. Figure 10 is a time-line which 
displays the important events which have occurred 


50 






Figure 9: DLA Organizational Chart 


























as DLA executes its pilot project. As can be seen by Figure 
10, the pilot has been rather time compressed. In a span of 
less than two years, DLA has had to create three performance 
plans, consider their position as a flexibility/ 
accountability pilot and write one performance report. 

2 2 Oct 9 3—r- Nomination submitted to OSD 

31 Jan 94- - Approval by OMB as a pilot 

17 Mar 94- - FY94 Performance Plan submitted 

Jun 94- - DoD Logistics Strategic Plan issued 

15 Sep 94- - FY95 Performance Plan submitted 
14 Nov 94- - Nomination for Waiver Pilot submitted 
2 Feb 95- - FY94 Performance Report submitted 
14 Apr 95—1— Fy96 Performance Plan due date 
Figure lO: Time-line for DLA Pilot Phase 

DLA's involvement in the GPRA pilot phase is another 
example of their top level management's dedication to "New 
Management" techniques. DLA has also been heavily involved 
with the National Performance Review (NPR) initiatives. As 
part of their Corporate and Performance Plans, DLA has 
identified fifty specific initiatives which are designed to 
improve the productivity and effectiveness of their core 
logistics operations. Several of these initiatives have 
been designated as reinvention laboratories under NPR. 

These initiatives are correlated with the Strategic Goals as 
outlined by the DLA Corporate Plan and are presented in the 
annual performance plans. 

Figure 11 contains the strategic goals and customer- 
oriented goals as they are laid out in the Corporate plan. 
The four strategic goals were derived from the DoD Logistics 
Strategic Plan of 1994. Thus, DLA's corporate plan is 
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DLA Strategic Goals: 


-Put customers first 

-Improve the process of delivering logistics 
support 

-Empower employees to get results 

-Meet customer readiness and weapon systems 

acguisition requirements at reduced cost 

DLA Customer-Oriented Goals: 

-Responsiveness 

-Timeliness 

-Quality 

-Operating Efficiency 
-Financial Performance 
-Customer Satisfaction 


Figure 11: DLA Corporate Goals 

Source; The DLA Corporate Plan, 1994. 

linked directly to the DoD's logistics strategy as one would 
expect it to be. DLA then takes these overall strategic 
goals one step farther and defines them in terws of what the 
customer desires from its services. These are the customer- 
oriented goals provided in Figure 11. Once the mission 
statement and overall goals were in place, DLA needed 
performance indicators to support the goals. 

B. PERFORMANCE MEAStTRES 

The GPRA requires that each agency "establish 
performance goals to define the level of performmce to be 
achieved by a program activity" and "establish performance 
indicators to be used in measuring or assessing the relevant 
outputs, service levels, and outcomes of each program 
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activity”. (P.L. 103-62, 1993) These measures are then to 
be provided in the annual performance plan. 

In order to comply with this requirement, DLA first had 
to identify the major activities in which it engages. Then 
they had to arrange these activities/functions by major 
business area. Figure 12 presents the results of this self- 
analysis. As can be seen, three or four primary activities 
were selected for each business area. The next step was to 
specify performance measures for each activity. 


Su pp l y 


Distribution Contract Mat 


Supply Mgt 
Fuels 

Reutilization 
National Stockpile 


Receiving/shipping 
Storage Operations 
Special Operations 


Preaward Admin 
Postaward Admin 
Contractor Perf 


Figure 12: DLA Activities 

Source: DLA FY95 Performance Plan, pg. 7 

As a convenient way of showing the relationship between 
the activity, performance measures and customer goals; DLA 
created an activity/measure matrix. The matrix is 
reproduced in Figure 13. On the horizontal axis, the major 
business area is identified along with the activities within 
that particular area. The vertical axis presents the 
performance measures broken down by the customer goal which 
it satisfies. Each performance indicator in the DLA 
performance plan supports one of the customer-oriented 
goals. For example, "ICP Processing Time", "Days to Close 
PQDR's" and "MRO/DRO Processing Time" are all performance 
measures which relate to the customer-oriented goal of 
"Timeliness". Also indicated are the two principal desired 
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logistic outcomes as identified by the DoD Logistics 
Strategic Plan of 1994. The grey shaded boxes are indicative 
of the performance measures which apply to the given 
activity (referred to as logistics operations by DLA). As 
an example, DLA considers "customer satisfaction" as an 
important measure for all activities. 



L3: Activity/Measurenent Matrix 
DLA FYSB Performance Plan, 199- 











As indicated by Figure 10, DLA has submitted two 
performance plans to date. A comparison of the performance 
measures contained in each plan gives some interesting 
results. Table 4 contains the results of this comparison. 
The Figure is arranged in similar fashion to Table 3 in 
chapter IV, except that the far left hand column is now the 
two DLA performance plans. 


Table 4: DLA FY94-FY95 Measurement Comparison 


DLA 

Total 

Input 

Output 

Effic¬ 

iency 

Effective¬ 

ness 

Outcome 

FY94 

22* 

0/ 0% 

8/ 36% 

6/ 27% 

8/ 36% 

0/ 0% 

Fy95 

24 

0/ 0% 

3/ 13% 

7/ 29% 

14/58% 

0/ 0% 


There are two significant results which can be observed 
in Table 4. First of all, DLA has shifted from several 
output measures to only a few in its second plan. Instead, 
the Fy95 plan contains a majority of effectiveness measures. 
Simple output (workload) measures are of limited use; 
however, when comparing them to standards or costs they 
become much more useful for managers. This is an indication 
of DLA's desire to shift more towards outcome type measures. 
However, the second result gleaned from this Table is the 
fact that DLA was still unable to include any "true" outcome 
measures within its plan. This testifies to the extreme 
difficulty in defining and capturing outcome impacts. 

The performance plan itself is arranged into three 
sections, one for each of the major business areas. Each 
section contains a description of the business area, the 
budget relationship, the associated performance indicators 
from Figure 13 which apply to the area, and definitions of 
the various performance measures. Figure 14 is an excerpt 
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from the FY95 performance plan. This particular business 
area is the Distribution function. The plan starts with a 
description of what all is involved in the distribution 
function. 

The next portion of the plan delineates the accounts 
(object-of-expense) located in the President's budget 
(Appendix, FY95) in which the area is funded. In this case, 
the distribution function uses part of the DBOF, Family 
Housing and MilCon line item funds. Additionally, a graphic 
displaying the portion of total DLA funding which is applied 
to the area is given. This value is in turn brokendown into 
the amount allocated to activities in the business area. 

The most important parts of the plan are the 
performance measures. The indicators are arranged by the 
customer-oriented goals. Also annotated with the measure is 
the activity to which it applies. These indicators are 
arranged into three columns. The first column gives a 
baseline value of the indicator if data had previously 
existed. (Some of the measures were brand new and no 
previous data was available for baseline determination.) The 
next two columns present the targets for the fiscal year of 
the plan as well as the following year. 

In this section the author analyzes the performance 
measures contained in DLA's performance plans. In doing so, 
several differences became evident between how the author 
and DLA categorize its performance measures. As mentioned 
in chapter IV, all agencies are to concentrate on measures 
of performance which gauge the outputs or outcomes of their 
major activities. DLA has indicated the "category" of 
measure it believes the individual performance measures fit 
in. Table 5 shows a comparison of how DLA categorizes its 
performance measures with how the author would categorize 
them. The author is using the definitions of performance 
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BUSINESS AREA: Distribution 

DESCRIPTION: Major functions of the Distribution business area include 
receiving and issuing materiel as directed by the managing inventory control points, 
care and preservation of materiel in storage, and other reimbursable services 
requested by the customer, such as unit and set assembly and assembly of 
deployable medical hospitals. 

BUDGET STRUCTURE AND RESOURCE REQUIREMENTS: 


Program and Finanrtng Schedule; 

97-'1930 0 <1.0.')l (Defense Business Operations Fund, partial), p 329 
97-0706-0'1-051 (Family Housing. Defense-Wide, partial) p. 32G 
97-0500-0-1-051 (Military Construction, Defense-Wide, partial), pp. 319-320 


FT95 Budget: (millions) 


$1,400 



Other Resource Requirements: No incremental requirements. Funded out of 
existing resources, DLA is pursuing a number of initiatives which are expected to 
enhance distribution logistics operations. 


PERFORMANCE 


RrsponnivcnrKK _^_ Baseline _ FYSS Tareet FY9e Target 

Indicator: Denial Bate 
Loeislice Opcrolionfs) 

Receiving and Shipping .74% <0,8% <0.8% 


Timeliness _ Dsissiine - FY 9S Tareet FY96 Tareet 

Indicator; Materiel Release Order Processing Time 
f xigi.stics Ooerationfa) 

Receiving and Shipping Not available < 1 day < 1 day high priority 

Not available <_7 days <G days routine 


Figure 14; Performance Plan Excerpt 

Source; DLA FY95 Performance Plan, 1994 
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Table 5: Comparison of Performance Measure Category 


Inputa/ 

Internal 


Effic- 


Outcomes 


measures as given by the Comptroller of the DoD in this 
analysis. (DoD, Comptroller, 1992) The categories are the 
same as those used in Figure 5 of chapter IV. 

Several differences can be noted from this Table. 

First of all, DLA classifies five of its measures as 
internal. Internal can be interpreted to mean inputs used 
within the operations of the business. For example, DLA 
classifies its performance measure "Space Utilization" as an 
internal (input) measure. Certainly, the amount of space 
available for storage is an input measure; however, DLA 
takes this measure one step farther by dividing the space 
occupied by the total usable space available. The author 
would describe this metric as a measure of effectiveness. 
Analysis of the other internal measures has similar results 
and therefore the author does not classify any of the 
measures as purely input measures. 

Another significant difference lies in the DLA term 
"Business Process Outcome". The author views these measures 
as simply outputs of the business operations. As an 
example, DLA classifies its measure "Material Release Order 
(MRO) Processing Time" as a business process outcome. MRO 
Processing Time is defined as "... the time period between 
when the depot receives the MRO and the time the material is 
offered to transportation for shipment.” (FY95 Performance 
Plan, 1994, pg.l6) This then is really a cycle time 
measurement and thus would be classified as an efficiency 
measure by the author. 
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The final major deviation between DLA's classifications 
and the author's is in the outcome measures. The primary 
mission of DLA is to provide the appropriate amount of 
logistical support to the services in times of war such that 
the services can complete their objectives successfully. The 
difficulty in measuring the performance of this "true" 
outcome becomes obvious. This measure could not be done on 
an annual basis. Measurement could not be completed until 
war had broken out and the logistical support was in fact 
sufficient. The next problem would be to ask how to define 
and capture the results of such a mission. Thus, the author 
believes DLA is not measuring the true outcome desired by 
their underlying primary mission. DLA identifies such 
measures as "Stock Availability", Product Conformance", and 
"Customer Satisfaction" as outcomes. However, the author 
would identify these measures as primarily efficiency and 
effectiveness measures. 

These differences are noted to show how difficult it is 
to come up with agreement on appropriate measures given the 
different perceptions of the individuals reviewing those 
measures. Additionally, one agency's outcome measure may 
simply be another agency's input or output measure. This 
relationship was evident in the ACC Maintenance Metric Index 
analyzed in chapter four. This measure was an outcome for 
the maintenance function at ACC; however, it was simply an 
output at the corporate ACC level. This is due to the 
broader mission scope ACC has as opposed to the maintenance 
function. 

The last section of the performance plan presents 
detailed definitions of the performance indicators. Also, 
a brief description may be given of how the measurement is 
derived. As a example, "Denial Rate" is defined as "A 
percentage based on the number of requisitions denied, in 
whole or in part, and the total number of requisitions 
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shipped (DLA Performance Plan, 1994, pp. 16-17) In other 
definitions some amplifying information may be given to help 
the reader understand the measure. In the case of the "MRO 
Processing Time" measure the following was added for 
clarity. "In accordance with DoD directives, we 
automatically downgrade to surface transportation all high 
priority MROs which do not have a Required Delivery Date or 
special project code, to realize transportation 
efficiencies This statement helped the reader understand 
how the MRO Processing Time was affected by DoD policies not 
directly under the control of DLA. 

Finally, a brief description is also supplied of how 
the above measures will be validated and compared to the 
targets. This fulfills the requirement for identifying the 
means of validating the measures contained in the 
performance plans as required by the GPRA. This function is 
accomplished by a performance measurement tracking system 
and is the subject of the next section of this thesis. 

C. PERFORHAMCE MEASUREMENT SYSTEM 

DIA has approached the above mentioned requirement by 
the use of an Executive Information System (EIS). In fact, 
one of the many stated reasons for DIA's entrance into the 
pilot project was to test its newly installed EIS system. 
This system compares actual results as they are recorded 
with the targets previously established by management. This 
comparison is performed in near real-time and is available 
to all policy, regional and operating managers for review. 
Each of the performance indicators contained in the 
performance plan is tracked by the EIS. This system also 
tracks other performance measures as well as the fifty 
management initiatives currently in progress throughout DIA. 

The EIS receives its inputs from various sources. Some 
of these sources include: direct manual input, internal data 
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bases such as the various business area management 
information systems, and external data bases such as AMIS 
the Air Force's contract data system. Figure 15 shows the 
data input sources for the Supply business area portion of 
the EIS. This portion of the system alone has 22 different 
input sources. In total, the EIS currently has 60 sources 
for its data with olans to add more over the next few vears 



As an example of how the EIS works, the path of one 
particular performance measure will be traced from its input 
sources to an EIS display screen. The measure selected was 
"Product Availability" for fuels. This is one of the 
indicators used in evaluating the Supply Management function 
of DLA. The customer-oriented goal of concern is 
"responsiveness" by DLA to a customer's request for fuel. 

The specific flow path for capturing the data used to 
calculate this measure is given in Figure 16. At the field 
level, reports are entered giving status of local fuel 
supply. Additionally, data on purchases and sales is also 
entered into the DFSC data base. This data is compared to 
the Inventory Management Plan. The plan contains the fuel 
stock required to fulfill estimated customer requests for 
the year. This comparison is performed by the EIS data base 
system along with such other calculations as averaging and 
variation computations. Finally, the data is ready to be 
displayed on the EIS. 



Figure 16: Product Availability Flow Path 

Source: DLA Corporate Office 
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Figure 17 is an example of an actual display screen 
printed from the EIS. Displayed here is the performance 
trend for Product Availability (fuels). There are several 
bits of information which can be observed on the display. 
First of all, a bar graph displaying the 1991-YTD94 values 
for product availability is given. As can be seen, data 
collection for this measure began between 1991 and 1992. A 
declining trend is also visible from 1992 to 1993. 

The next graphic to the right is a trend analysis of 
aggregate product availability for years 1992 and 1993. 

Along with the aggregate line, a line depicting the goal is 
included. Calculations of the mean and variation are 
presented as well. From this graphic, a manager might be 
able to recognize seasonal fluctuations, compare the actual 
results to the target, determine how much the measure 
varies, or special occurrences which may have affected the 
status of fuel availability. Figure 17 suggests that a 
large decline in fuel available occurred between the months 
of November 1992 and September of 1993. Therefore, the 
system has identified that "something" has occurred within 
that time period, allowing the manager to figure out what. 
The remaining two graphics help the manager do this. 

One of the more informative capabilities of the EIS is 
the ability of the system to break down measures by CINC, 
program or region. The bottom two graphs of Figure 17 show 
product availability broken out by the major CINCs. This 
allows the manager to take his/her analysis one step farther 
and perhaps identify what region (or in this case, CINC) has 
contributed to the trends indicated above. As an example, 
the manager may decide to track down why the Southern 
Command (SOUTHCOM) has had such erratic results for this 
particular measure. (Perhaps extensive contingency 
operations have caused the fuel supply to dwindle more 
rapidly than normal.) 
















Across the very bottom of the display screen is a 
selection bar. This bar allows the manager to select 
whichever performance measure he/she may be interested in. 
There is also a means to enter the initiative tracking 
system contained in the EIS as well. (This system tracks the 
status of the fifty managerial initiatives currently in 
progress at DLA.) There are many other features of the EIS 
which are not discussed within the context of this thesis 
but are useful to the managers at DLA. 

The analysis of DLA's measurement system is not meant 
to provide a specific model upon which to base other agency 
systems. Obviously, each individual agency will need its 
own specific system, given the unique qualities of the 
organization. The point is that agencies will need to have 
accounting systems and a means for capturing performance 
measurements. A cost accounting system capable of producing 
unit costs is essential for deriving appropriate efficiency 
measures. 

D. EVALUATION 

This section describes how the individuals at DLA feel 
the process of implementing GPRA is working to date. 

Included within the text of the section are strengths, 
weaknesses, difficult items to accomplish and finally, items 
yet to be accomplished. In formulating this section, 
several individuals from each business area were 
interviewed. Both corporate level and field activity level 
employees were included. 

As with any new process, individuals working with the 
process will be able to identify many strengths and 
weaknesses associated with it. GPRA implementation is no 
exception to this rule. Thus in evaluating the process, 
several individuals were asked to explain what they thought 
might be the most pronounced strengths and weaknesses of the 
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system. In each case, several similar responses were given. 

Strenahts : The primary strengths as indicated by the 

employees are given below. 

• Customer satisfaction is now looked at; a survey is 
being used to find out just what our customers desire 

• Performance measurement will now be used as a means 
of managerial evaluation; regional, field activity 
and depot included 

• Aligns the efforts of the field activities to 
specific goals as set forth by the corporate office 

• We are able to quantify things never before 
measurable; e.g., lead times, total asset 
availability... 

• Vast amount of information has become available to 
managers at all levels 

• Surfaces problems, not the symptoms of problems 

• Coordination of vision all the way to the bottom of 
the organization; it is a systematic means of doing 
so 

It became evident from the responses that customer 
concerns were the primary driving force behind DLA 
operations. One hundred percent of the individuals 
interviewed identified customer satisfaction measures as 
being a major strength of the new performance measurement 
system. Some of the customer oriented measures being used 
are "customer price change", "customer complaints", and an 
overall "customer satisfaction index" for each of the 
business areas. The attainment of this last measure is a 
major task in and of itself. DLA is currently involved with 
a survey of over 32,000 of its customers. This survey is to 
measure the baseline of customer satisfaction with DIA's 
services and will be used to set the targets for future year 
measurements. 
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Another major, and rather surprising response came from 
the field activity level managers. They said that having 
their own performance evaluations being based at least in 
part on activity performance measures was beneficial. In 
many cases, changing the standards by which managers are 
evaluated intimidates them and invokes a negative response. 
This is evident in the considerable amount of time spent in 
trying to develop a military fitness-report/evaluation 
system which would be deemed fair and eguitable to those 
evaluated. This has been going on for many decades in the 
DoD. But in the interviews conducted for this thesis, most 
managers responded favorably to the change in evaluation 
system. Most felt this would be a more fair and equitable 
system than the current one, although concerns about exactly 
how the system would work were also evident. 

A shared sense of vision from top to bottom was also 
mentioned as a benefit of this process. The system provides 
not only a sense of vision, but also a systematic means of 
coordinating the goals of the upper management. Simply put, 
the system gives "concrete" evidence of how the organization 
is performing. This evidence is visible to all individuals 
interested, and is displayed throughout the agency on its 
Executive Information System. Additionally, it is supplied 
on the performance reports and plans produced annually. 

Many workers become frustrated when they cannot see the 
results of their work in response to lofty goals set on 
high. Several of the individuals interviewed claimed they 
now have at least some idea of how their activities are 
performing with respect to the goals of the corporation. 

The last major strength of this process stems from the 
system's ability to quantify data. Two of the three 
business areas identified the fact they can now quantify 
data/measures for which they previously only had a 
"feeling". A specific example mentioned was various lead 



times. Previously, no systematic means existed for 
measuring the time it took to complete a specific activity. 
Now several lead/cycle time measurements are included in the 
performance plan for DLA. Examples include: MRO Processing 
Time, Logistics Response Time, and Days to Close PQDR's. 

Now that an automated system is available, these times can 
be measured rather easily. 

Weaknesses: Interviewees also identified several 

weaknesses associated with their performance measurement 
process. The major concerns are provided below. 

• Lack of automation for all performance measures; 
i.e., a large amount of data is still entered by hand 

• Too many measures to start with, would be better if 
we had just one measure for each activity 

• Data integrity, timeliness 

• Performance measure definitions are not all perceived 
the same; i.e., keep them very simple 

• Incompatibility among the various information systems 

• Traditional measures still being used in many cases; 
(especially in supply management) many measures used 
are not a concern of the customers 

The first major weakness which will be discussed deals 
with the type of measures reported. According to the 
interviewees, too many non-customer oriented measures are 
still being used (i.e., internal/input measures). An 
example might be the "space utilization" measure. While 
this is certainly important to the managers at DLA, 
customers probably would not be interested in how space is 
used at DLA. Additionally, stakeholders in the process such 
as plan reviewers are more concerned with the overall 
objectives of the organization as opposed to internal 
measures. The general consensus among interviewees was that 
while significant strides had been made at shifting towards 
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customer oriented goals and measures, more work still needed 
to be done. 

The next weakness is related to the previous one. 
Responses indicated that many employees thought too many 
"traditional" measures were being used, or that too many 
measures in general were provided. The Supply Management 
business area seemed to feel the strongest about this 
weakness. However, a comparison of the FY94 and FY95 
performance plans conveys results to the contrary. Table 6 
displays this data. The Table shows the three business 
areas and the number of measures deleted from FY94 plan as 
well as the number of those added in the FY95 plan. Of the 
ten FY94 supply management measures, only five survived to 
the FY95 plan. Additionally, six new measures were added. 
Results for the other two areas are similar. In the case of 
the Contract Management, a wholesale change-out of measures 
occurred. Thus, it would appear that DLA is shifting away 
from some of their traditional measures. 


Table 6: Performance Measure Comparison 


FY94 to Fy95 

Supply Mgt 

Distribution 

Contract Mgt 

Deleted 

5 

4 

3 

Added 

6 

3 

7 

Total FY95 

11 

8 

7 


Where sheer volume of measures is concerned, twenty- 
four measures for an agency with -58,000 employees could be 
arguably appropriate. This gives each of the ten primary 
activities (see Figure 13) an average of 2.4 measures each. 
This does not seem to be an unreasonable amount. Thus the 
data do not necessarily suggest that too many measures are 
being used in this case. 
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The third weakness indicated is the incompatibility of 
many of the information systems associated with DLA. This 
problem stems from the fact that DLA underwent a major 
consolidation effort at the start of the 1990's. Base 
closure, downsizing, and rightsizing all have helped lead to 
DLA absorbing the logistics functions of the three services. 
When this occurred, DIA inherited many different information 
systems and data bases which were not compatible with DIA's 
systems. This is a problem associated with many agencies 
which are undergoing consolidation processes. DIA is 
currently engaged in integrating these various systems via 
its "process" strategic goal initiatives. 

The final weaknesses stems in part from the previous 
problem. Many interviewees indicated their concern with 
data integrity and timeliness. This problem originates in 
the fact that many inputs to the EIS are still made by hand. 
This is a result of the incompatibility among the systems 
previously mentioned. Certainly, many performance 
measurement systems have problems with data integrity. 
Whether or not incorrect data is entered purposefully or by 
mistake, erroneous data will show up. This problem will be 
alleviated somewhat when all the information systems are 
integrated. Still at some point data will have to be 
manually entered. 

Most Difficult Item; These weaknesses are examples of 
just how difficult it can be to introduce a new process into 
an organization. One of the questions asked of the 
individuals interviewed was what they thought was the most 
difficult item to achieve to date in this implementation 
process. Three overwhelming responses were evident. These 
responses are provided below. 

• Convincing the field activities that this is for 
real, not just another program, it is the law 
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• Getting reliable data from systems that don't 
communicate together 

• Developing appropriate measures for the goals 
expressed by upper management 

The last two problems were identified as weaknesses 
and were discussed earlier. The first item is perhaps the 
hardest of all to accomplish. This requires a change in 
employees' beliefs, not just hardware connections or 
resources to purchase needed items. The GPRA is in fact a 
law whose major thrust will require action by all federal 
agencies in 1997. As mentioned earlier in this thesis, this 
subject appears to cross party lines in Congress and 
therefore does not appear to be in danger of being repealed 
anytime in the near future. The sooner agencies are able to 
convince their personnel of this fact the better equipped 
they will be to deal with the requirements of this law. 

Future Problem: Another question asked of the 
interviewees was what they thought remains the most 
significant problem to overcome in the future. The response 
given was the transition from the current budget system to a 
performance budget system. GPRA does not provide for 
performance budgeting within the law; but, the long term 
goal of GPRA seems to be aimed at using performance 
budgeting for the federal budget process. The basic 
question here was how the performance measures were going to 
translate into a resource allocation tool. 

DLA has started to include GPRA performance reporting 
into their Planning, Programming, Budgeting and Execution 
(PPBE) cycle. In Fy95's PPBE schedule memorandum, outlined 
by the Executive Director of Strategic Programming and 
Contingency Operations, performance was a primary issue to 
be considered. This was evidenced by the significant 
emphasis the director placed on performance as indicated in 
the quote from the schedule. 



We must use the PPBE process to maintain the 
momentum you have put in motion to reduce customer 
costs, improve performance, and achieve DLA's 
strategic goals. Your continued involvement in 
the process assures DLA will be the DoD example 
for a disciplined, performance-based process in 
all elements of PPBE. (Gallo, 1995) 

According to the PPBE memorandum, future plans will 
incorporate the performance measures associated with GPRA. 
This will be accomplished via Primary Level Field Activity 
(PLFA) plans. The activity level plans will document the 
PLFA goals and objectives. The activity commanders will 
then be held accountable to these goals in that the plan 
becomes a performance contract between the activity 
commander and the corporate office. This then becomes the 
PLFA's stated compliance with GPRA and is the basis for the 
PLFA programming and budget requirements. The PLFA plans 
are then aggregated into the business area plans for the 
fiscal year. Business area plans in turn should directly 
support the DLA Corporate Plan. This makes up the bulk of 
the planning portion of the PPBE cycle for DLA. 

Figure 18 is a graphic representation of DLA's 
integrated PPBE schedule. This graphic shows the timing 
associated with the DLA system in order to meet the external 
requirements placed on them by OUSD(A£iT). (DLA submits 
their budget request via the Office of the Under Secretary 
of Defense for Acquisition and Technology) The performance 
plans under GPRA will be submitted to the OUSD(C) in August 
of the year in which the budget will be prepared. 

(Currently, performance plans for the DoD are being handled 
by the DoD Comptroller. Whether or not this continues after 
full implementation is in question.) For example, the 
performance plans for FY97 will be submitted to the 
Comptroller office in August of 1995. This will facilitate 
use of the performance plans as a means for budget 
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consideration. The performance reports will be submitted in 
January of the year immediately following the fiscal year of 
the associated performance plan. This will hopefully 
provide ample time for review by OUSD(C) and 0MB in order to 
allow for feedback to the agencies prior to the next year's 
plan submission. 

The Programming portion of the DLA PPBE system consists 
of presentations of the Business Area Plans by the business 
area managers. The presentations are an assessment of the 
future performance goals, resource requirements, and 
programs needed to achieve performance goals. (Gallo, 1995) 
Since the resources for DLA are controlled primarily by 
customer needs (DBOF activities are funded by customer 
estimates of required services), the Business Area managers 
must consider tradeoffs between resourcing levels and 
performance projections. This then is an assessment of the 
risk in achieving performance goals given a constraint in 
resources. 

DLA seems to be well on its way to including 
performance measures within the PPBE system. The problem 
which needs to be addressed is how these performance 
tradeoffs will be incorporated into the budgeting portion of 
the process. As noted earlier in this thesis, the 
President's Budget is broken down by object-of-expense. 
However, performance based budgeting relies on activities as 
a base for fund allocation. The goals and performance 
measures used by DLA in implementing GPRA are based upon the 
activities which they perform. Since there is no direct 
correlation with the funds available in the budget and the 
activities DLA performs, a gap occurs when considering how 
much funding should be applied to where. This is the major 
concern of those who responded to the question of the most 
difficult item yet to be accomplished. 


74 



lntegra%WWm'^f^'’Schedule 


Strategic 

Plannirjg 


Program 

Development 


Budget 

Formulation 


Performance 

Evaluation 


Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep 



Figure 18: DLA PPBE Schedule 

source: PPBE Memorandum, 1995 





E. CONCLUSION 


This chapter has described and evaluated the process by 
which DLA is implementing its GPRA pilot project. This 
agency was chosen because of its two plus years of 
experience with GPRA. The chapter described the performance 
measures used by DLA. It then went on to describe the 
systematic means by which these measures are captured. 
Finally, a look at how the individuals who operate within 
the process view the process was also presented. The 
purpose of the chapter was to provide other agencies with 
ideas of how to accomplish this task. Lessons learned by 
DLA can be useful to others in their attempts at 
implementing GPRA in 1997. 

As seen in the chapter, performance plans/measures will 
undoubtably change several times before a final set of 
measures is decided upon. In DLA's case, over 66 percent of 
the FY95 plan measures were not part of the FY94 plan. 
Similarly, the plan itself shrunk from almost 120 pages down 
to just 24 pages. This should warn agencies who will engage 
in GPRA plan writing that plans and measures are not easily 
generated. This is a process which will probably take 
several iterations to complete. 

Secondly, the performance measurement system developed 
by DLA was not meant to be a model for other agencies to 
follow. On the contrary, this system was specific to DLA 
only and would not work for any other agency as is. This 
analysis was performed to show that DLA already had a 
performance measurement tracking means available to them. 
Many other agencies will not have this luxury; as a 
consequence they may take significantly longer to develop a 
means for validating and monitoring performance. Good cost 
accounting systems and performance tracking systems are a 
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must if performance measurement is to impact resource 
allocation significantly. 

Several strengths and weaknesses of the GPRA 
implementation process were also identified within the 
context of this chapter. Managers at DLA seemed pleased 
that customer satisfaction was becoming a primary driving 
force for DLA. However, they felt more measures should be 
directly linked to customer concerns. Managers also 
appeared motivated by the fact that personal evaluations 
would be based upon the performance measures for the 
activity over which they held control. Many employees felt 
a sense of vision concerning performance now permeates the 
organization. And finally, data never before captured is 
now being used daily to measure performance. 

On the other hand, several measures still require hand 
input according to the managers. Therefore, considerable 
concern over data integrity and timeliness was indicated at 
the corporate office levels. Additionally, many members 
felt that "traditional" measures were still being used far 
too often. The data analysis by the author suggested that a 
shift away from traditional measures appears to be in 
progress despite the feeling of those interviewed. Perhaps 
the most significant difficulty faced by the administration 
at DLA is motivating its workers to realize GPRA is not iust 
another program . 

Lastly, considerable concern as to how GPRA performance 
measurement will be reflected in resource allocation has 
been voiced at DLA. DLA has attempted to bridge this gap by 
using the Activity Level Performance Plans as a performance 
contract with the corporate office. While this is an 
internal fix for DLA, the federal government still has 
significant problems with resource allocation between 
agencies without an activity based budgeting format. 
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Chapter VI will provide the lessons learned from this 
and previous chapters as well. Recommendations for how 
other DoD agencies can implement GPRA will be provided. 
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VI. CONCLUSIONS/RECOMMENDATIONS 


A. RECOMMENDATIONS 

This section provides the recommendations by the author 
based upon the research conducted in writing this thesis. 

In general these recommendations are directed at agencies 
who will prepare performance plans under the auspices of 
GPRA in the near future. 

1. Agencies should start working now . Several aspects 
of performance planning will take considerable time to work 
out. As shown in the case of DLA, subsequent performance 
plans created may not look anything like the initial one. 

DLA changed 66 percent of its performance measures in the 
first year alone after realizing the measures were not 
appropriate. Additionally, the sheer size of its plan was 
significantly reduced over the two years. These plans will 
be required for submittal in September of 1997. DoD will 
need to decide at what level these plans will be required. 
Will the plans be directly linked to the budget submittal 
and thereby be required at every level, or just the 
corporate agency level? 

2. Agencies should keep the plans_simple. Several 
examples throughout the thesis were given to show that 
simplicity seems to be the best way to approach GPRA 
performance plans. Large convoluted measures which would be 
difficult for an outside administrator to understand are not 
beneficial in gauging performance. Moreover, verbose 
explanations of future GPRA implementation plans or of items 
not directly related to the measures themselves should not 
be provided. The plans should state the mission and vision 
of the organizations. The measures, their targets, baseline 
data if available, and perhaps a long-term goals should be 
identified as well. Definitions of the measures should be 
provided for clarity. Finally, a means for validating the 
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measures as well as their relationship to the budget is also 
required by GPRA. These plans will be aggregated all the 
way to a federal government level by the OMB. If the plans 
are as large and complex as some of the pilot plans, DoD and 
OMB will be faced with a daunting task in formulating a 
reasonable plan from the subordinate plans. 

3. Agencies should first identify the primary 
activities in which they engage. This in-turn will help 
identify the measures of performance to be used in gauging 
these activities. As DLA did when it created its second 
plan, deciding upon the activities will help clarify just 
what measures to look at. Additionally, if performance 
budgeting is the long-term goal of GPRA, activities will be 
used as the basis for the budgets. In the meantime, 
identifying the activities in which the organization engages 
will be beneficial to the managers with oversight. 

4. Some of the examples contained within this thesis 
should be used for benchmarking. This is especially true 
when it comes to displaying the measures in annual plans. 

Of all the plans reviewed for this thesis, DLA had the most 
easily understood format in its FY95 plan. Moreover, this 
plan was fairly simple and concise at 24 pages. Arranging 
the plans in a similar fashion as Figure 2 and Figure 14 
makes them easily understood and allows for trend analysis 
over a range of years. The particular types of measures 
chosen by the pilots may also fit into other organization 
plans. 

5. Agencies should r ealize their measures may not be 
interpreted as expected. As shown in this chapter, DLA 
feels that a great number of its measures are of the outcome 
type. The author, on the other hand, takes a broader view 
of outcomes and therefore was in disagreement with DLA on 
several measures. Based on this analysis, outcomes were by 
far the most difficult to capture. Very few agencies were 



able to include true outcome measures in their plans. 
Moreover, outcome measures may not even be measurable on an 
annual basis. Some of these outcomes take years to achieve 
depending on the orientation of the agency. For example, 
one of DLiA's primary missions is to provide adequate 
logistics support to the services in times of war to ensure 
a successful campaign. This outcome goal could only be 
measured in retrospect after a conflict had occurred. 

6. Agencies should concentrate on measures of 
efficiency and effectiveness. While outcome measures should 
be included if possible, efficiency and effectiveness 
measures are more attainable and should be used as much as 
possible. Simple measures of output are not nearly as 
useful as efficiency or effectiveness measures are. Input 
measures are not required by GPRA and in general are not of 
interest to outside stakeholders. Many examples of these 
measures are contained within this report. 

7. Good accounting and performance measurement systems 
are reguired to implement GPRA in an efficient manner. DLA 
already had an Executive Information System in place when it 
volunteered to act as a pilot project. This system is an 
excellent way to monitor performance throughout the 
organization. Agencies will need both types of systems in 
order to capture the data necessary for GPRA implementation. 
Therefore, agencies should start looking at how to 
accomplish this task as early as possible. Also indicated 
in the analysis of DLA's system was the fact that it is 
still changing to suit DLA's needs. 

8. Agencies should link together several initiatives 
currently in pro gress within t he DoD. GPRA fits neatly into 
the initiatives such as Total Quality Leadership/Management 
and the National Performance Review. Most of the pilot 
projects used GPRA as a means to enhance initiatives already 
in progress in its organization. For example, the Army Corp 
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of Engineers entered into the pilot phase as an extension of 
its National Operation and Maintenance Program Plan of 
Improvement. The tools contained within TQL/M would benefit 
managers as they attempt to create performance indicators 
and plans. For example, the Delphi method could be used to 
establish the primary activities the organization wishes to 
measure. Additionally, agencies should also use current 
goals and statements as a starting point. For example, the 
mission, vision and primary activity goals for the 
Department of the Navy are already contained in its 
"Forward... from the Sea" document. Therefore it would be 
inefficient to start the process from scratch when some of 
the data required is already in existence. 

9. Agencies should look at how the performance plans 
would be linked to resource allocation. Perhaps the most 
difficult item to complete in all of GPRA implementation 
will be to find how the measures will be used to allocate 
resources. DLA has started this process by creating a 
performance contract between its field activity managers and 
the corporate office. Whether or not this could occur on a 
federal or even DoD level is still subject to question. The 
primary stumbling block at the federal level is the object- 
of-expense base currently used in the federal budget 
process. 

10. Finally, agencies should realize that GPRA 
implementation is not ~iust a budget drill. Performance 
measurement is beneficial to the managers who run the 
organization. Moreover, several uses for performance 
measures can be given. A study by the Urban Institute 
suggests nine potential uses for performance measures (Hatry 
et al, 1977, pp. 195-199). 

• Reviewing the progress and trends of government 
services. 
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• Guiding resource allocation decisions... by area, 
clientele. 

• Helping support budget preparation and justification. 

• Contributing to in-depth program evaluation and 
analysis. 

• Developing performance incentives emphasizing quality 
for government employees. 

• Controlling the quality of performance for services 
contracted out. 

• Checking efficiency measures for quality. (This is a 
function of effectiveness measures specifically.) 

• Providing a managerial control system for resource 
reallocation. 

• Improving communication between government and 
citizens... through surveys, observers, complaint 
bureaus. 

These functions were originally discussed as those 
provided by effectiveness measures only. However, the 
author feels these functions are provided by performance 
measurement in general. As can be seen by the above list, 
budget formulation is an important part of performance 
measurement, but not the only part. Managerial and program 
control are greatly enhanced by measuring the performance of 
activities in which a program engages. 

These suggestions provided are by no means all 
inclusive, other results of this analysis may help agencies 
in developing future performance plans under the auspices of 
GPRA. The examples provided in this study are the first to 
be developed under GPRA and most certainly will be improved 
upon as more subsequent plans are created. 
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B. FUTURE RESEARCH SUGGESTIONS 


Since this process is so new, agencies will undoubtably 
be searching for help on establishing plans. For this 
reason, the analysis contained within this thesis is only a 
starting point from which further research can be launched. 
The following are a few suggestions for future research in 
this area. 


• A comparison between the first plan submitted and the 
second plan submitted for all the pilots could be 
completed when the FY96 plans are finished in much 
the same way as the DLA FY94 and FY95 plans were 
compared in this thesis. 

• Significant research needs to be completed on how the 
performance plans under GPRA could be linked to the 
DoD PPBS budget process. Moreover, a link to the 
federal budget process is also needed. 

• An analysis of how to define and capture outcome 
measures for DoD agencies is desperately needed. 

• A look at how these individual agency plans might be 
aggregated into a DoD corporate plan would be 
beneficial as well. Additionally, this could be 
taken one step farther and a federal wide aggregation 
could be analyzed as well. 

In summary, this chapter provided as a means to direct 
the reader to the important portions of the chapter. A list 
of recommendations based upon the results of this analysis 
were provided. These will be helpful to agencies which will 
start creating performance plans in the near future. 

Several examples were given as a source of benchmarking for 
the agencies looking towards GPRA implementation. Lastly, 
some areas for further research were presented as a possible 
source for future theses. 
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